Algebraic Identifiability of Gaussian Mixtures

We prove that all moment varieties of univariate Gaussian mixtures have the expected dimension. Our approach rests on intersection theory and Terracini's classification of defective surfaces. The analogous identifiability result is shown to be false for mixtures of Gaussians in dimension three and higher. Their moments up to third order define projective varieties that are defective. Our geometric study suggests an extension of the Alexander-Hirschowitz Theorem for Veronese varieties to the Gaussian setting.Comment: 18 pages, to appear in International Mathematics Research Notice

Moment Identifiability of Homoscedastic Gaussian Mixtures

We consider the problem of identifying a mixture of Gaussian distributions with same unknown covariance matrix by their sequence of moments up to certain order. Our approach rests on studying the moment varieties obtained by taking special secants to the Gaussian moment varieties, defined by their natural polynomial parametrization in terms of the model parameters. When the order of the moments is at most three, we prove an analogue of the Alexander-Hirschowitz theorem classifying all cases of homoscedastic Gaussian mixtures that produce defective moment varieties. As a consequence, identifiability is determined when the number of mixed distributions is smaller than the dimension of the space. In the two component setting we provide a closed form solution for parameter recovery based on moments up to order four, while in the one dimensional case we interpret the rank estimation problem in terms of secant varieties of rational normal curves.Comment: 27 pages, 1 table, 1 figur

Moment Varieties of Gaussian Mixtures

The points of a moment variety are the vectors of all moments up to some order of a family of probability distributions. We study this variety for mixtures of Gaussians. Following up on Pearson's classical work from 1894, we apply current tools from computational algebra to recover the parameters from the moments. Our moment varieties extend objects familiar to algebraic geometers. For instance, the secant varieties of Veronese varieties are the loci obtained by setting all covariance matrices to zero. We compute the ideals of the 5-dimensional moment varieties representing mixtures of two univariate Gaussians, and we offer a comparison to the maximum likelihood approach.Comment: 17 pages, 2 figure

Likelihood geometry of correlation models

Correlation matrices are standardized covariance matrices. They form an affine space of symmetric matrices defined by setting the diagonal entries to one. We study the geometry of maximum likelihood estimation for this model and linear submodels that encode additional symmetries. We also consider the problem of minimizing two closely related functions of the covariance matrix: the Stein's loss and the symmetrized Stein's loss. Unlike the Gaussian log-likelihood these two functions are convex and hence admit a unique positive definite optimum. Some of our results hold for general affine covariance models

Convex Hulls of Curves: Volumes and Signatures

Taking the convex hull of a curve is a natural construction in computational geometry. On the other hand, path signatures, central in stochastic analysis, capture geometric properties of curves, although their exact interpretation for levels larger than two is not well understood. In this paper, we study the use of path signatures to compute the volume of the convex hull of a curve. We present sufficient conditions for a curve so that the volume of its convex hull can be computed by such formulae. The canonical example is the classical moment curve, and our class of curves, which we call cyclic, includes other known classes such as $d$-order curves and curves with totally positive torsion. We also conjecture a necessary and sufficient condition on curves for the signature volume formula to hold. Finally, we give a concrete geometric interpretation of the volume formula in terms of lengths and signed areas.Comment: 15 pages, 5 figures. Comments are welcome

Estimating Gaussian mixtures using sparse polynomial moment systems

The method of moments is a statistical technique for density estimation that solves a system of moment equations to estimate the parameters of an unknown distribution. A fundamental question critical to understanding identifiability asks how many moment equations are needed to get finitely many solutions and how many solutions there are. We answer this question for classes of Gaussian mixture models using the tools of polyhedral geometry. Using these results, we present an algorithm that performs parameter recovery, and therefore density estimation, for high dimensional Gaussian mixture models that scales linearly in the dimension.Comment: 30 page

Varieties of Signature Tensors

The signature of a parametric curve is a sequence of tensors whose entries are iterated integrals. This construction is central to the theory of rough paths in stochastic analysis. It is here examined through the lens of algebraic geometry. We introduce varieties of signature tensors for both deterministic paths and random paths. For the former, we focus on piecewise linear paths, on polynomial paths, and on varieties derived from free nilpotent Lie groups. For the latter, we focus on Brownian motion and its mixtures.Comment: 52 pages, 1 figure, 6 tables; to appear in Forum of Mathematics, Sigm